On the Rate of Convergence to the 
Marchenko— Pastur Distribution 



F. Gotze 

Faculty of Mathematics 
University of Bielefeld 
Germany 



A. Tikhomirov 

Department of Mathematics 
Komi Research Center of Ural Branch of RAS, 
Syktyvkar state University 
Syktyvkar, Russia 



II 



Abstract 

Let X = {Xji^) denote nxp random matrix with entries Xji^, which are indepen- 
dent for 1 < j < n,l < k < p. We consider the rate of convergence of empirical 
spectral distribution function of matrix W = |XX* to the Marchenko-Pastur 
law. We assume that EXj^ = 0, EX|^ = 1 and that the distributions of the 
matrix elements Xjk have a uniformly sub exponential decay in the sense that 
there exists a constant x > such that for any l<j<n, l<k<p and any 
i > 1 we have 

Pv{\Xjk\ >t} < x^iexp{-t^}. 

By means of a recursion argument it is shown that the Kolmogorov distance 
between the empirical spectral distribution of the sample covariance matrix 
W and the Marchenko-Pastur distribution is of order 0{n~^ log^n) with some 
positive constant b > 0. 



1 Introduction 



Consider a family of independent random variables X = {Xj^}, 1 < J < 1 < < 
defined on some probability space Pr). Let X = (Xj^) be matrix of order nxp 

and let W = ^XX*. Denote by {si, . . . , s^} the eigenvalues of the matrix W and 
introduce the associated spectral distribution function 

J^n{x) = — card {j < n : < x}, x G M. 
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tribution. 
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Averaging over the random values Xij{u), define the expected (non-random) empir- 
ical distribution functions 

F„(x) =EJ-„(x). 

We assume that p — p{n) and lim^_!>oo ■r = ?/ e (0, oo). Without loss of generality we 
shall assume that y G (0,1]. Let Gy{x) denote the Marchenko-Pastur distribution 
function with density gy{x) = G'y{x) = 2^\/{x - a){h - x)I[a,b]{x), where I[a,h]{x) 
denotes the indicator-function of the interval [a, 6], a = (1 — ^/yY, h = {1 + y/y)"^. 
We shall study the rate of convergence J^n{x) to the Marchenko-Pastur law assuming 
that Pr{|Xjfc| > t} < x^^exp{— t^} for some x > 0. This problem has been studied 
by several authors. In particular, we proved in [10] that the Kolmogorov distance 
between J-'„(x) and the distribution function Gy{x), A* := sup 3, |J-'„(x) — Gy{x)\ is of 
order Op(n~2). Bai et al. showed in [1] that A„ := sup 3, |-F„(x) — Gy{x)\ = 0{n~^). 
For the Laguerre Unitary Ensemble we proved in |3] that A„ = 0{n~^). Let ?/ = ^ G 
(0, 1] in the what follows. For any positive constants a > and x > define the 
quantities 

log n (log log n)" and Pn ■= {ln,a)^^'^ ■ (1-1) 
The main result of this paper is the following 

Theorem 1.1. Let EXjk = 0, EXj^, = 1 and there exists a constant x > such that 
for any 1 < j < n and 1 < k < p and any x > 1, 

Pr{\Xjk\ >x}< x"^ exp{-x^}. (1.2) 

Then for any a > there exist a positive constants C and c, depending on x and a 
such that 

Pr{sup \Tn{x) - Gy{x)\ > n-^(3l} < Cexp{-c/„,„}. (1.3) 

X 

We apply the result of Theorem 11.11 to investigation of eigenvectors of the ma- 
trix W. Let Uj = (wji, . . . ,Ujn)'^ be eigenvector of the matrix W corresponding to 
eigenvalue s'j, j = 1, . . . , n. We prove the following result. 

Theorem 1.2. Under condition of Theorem for any a > there exist constants 
C, c, depending on x and a such that 

Pr{ max > — } < Cexp{-c/„Q,} (1.4) 

l<j,k<n n ' 

and 

Pr{ max I >^ kivP 1 > —^1 < C expj— c/„ a\- (1-5) 

We use a relatively short recursion argument based on the approach developed in 
[5] and [6] and ideas similar to those used in Erdos, Yau and Yin [8], Lemma 3.4. 
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2 Proof of the main Theorem 

To bound A* we shall use an approach developed in Gotze and Tikhomirov, [10] and 
[5]. We shall apply a bound of the Kolmogorov distance between distribution func- 
tions via distance using their Stieltjes transforms. We denote the Stieltjes transform 
of J^n{x) by mn{z) and Stieltjes transform of the Marchenko-Pastur law by Sy{z). 
We shall use the "symmetrization" of the spectrum sample covariance matrix as in 
[5] . Introduce the [p + n) x [p + n) matrix 



VP 



O X 

X* o 



(2.1) 



where O denotes a matrix with zero entries. Note that the eigenvalues of the matrix 
V are ±si, . . . , ±s„, and with multiplicity p — n. Let R = R(-2) denote the resolvent 
matrix of V defined by the equality 

R= (V-^Wp)-\ 

for all z = u+iv with v ^ 0. Here and in what follows 1^ denotes the identity matrix of 
order k. Sometimes we shall omit the sub index in the notation of the identity matrix. 
It is well-known that the Stieltjes transform of the Marchenko-Pastur distribution 
satisfies the equation 

yzs\z) + {y-l + z)sy{z) + 1 = 

(see, for example, equality (3.9) in [1]). If we consider the Stieltjes transforms 'Sy{z) 
of the "symmetrized" Marchenko-Pastur distribution Gy{x) = ^+'^is°^'^«(^ ) ^ then it 
is straightforward to check that 'syi^z) = zsy{z^) and 

ys^{z) + {^^^ + z)sy{z) + l = Q. (2.2) 

(see Section 3 in [5]). Furthermore, for the Stieltjes transform of the "symmetrized" 
empirical spectral distribution function Fn{x) = i+'^'g'^^-^"(^ ) have 

n+p 

y 



n ^ — ' n ^ — ' yz 

j=l j=n+l ^ 

(see, for instance. Section 3 in |5j). In the what follows we shall consider symmetrized 
random values only. We shall omit the symbol ^ in the notation of the distribution 
function and its Stieltjes transform. Let Tj = {1, . . . ,n} \ {j}. For j = 1, . . . ,n, 
introduce the matrices V^-'^ obtained from V by deleting the j-th row and j-th 
column, and define the corresponding resolvent matrix R*^-'^ by the equality R*--^-* = 
(V0-) - zU,^,)-\ Let m^^\z) = i E^^t, 
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We shall use the representation, for j = 1, . . . , n, 

\ 

(see, for example. Section 3 in ^). We may rewrite it as follows 

where = Sji + + ^js and 

1 ^ 1 

^ k=l l<kT^l<P 

This relation immediately implies the following two equations 



Rjj — ZIT — / ~ ZT 1~ 

[z + ymn[z) + z + ymn{z) + ^ 



and 



1 1 1 r. 

m„{z) = 7- H > EiRii 

^ ' z + ym^iz) + ^ iz + ym^{z) + V--^) n j^^ ' " 



2 n 



z + ymn{z) + ^ {z + ymn{z) + ^yn^^^ 

11^" 1 1 " 

+ {z + ym4z)yn g ^ + z + ym^{z) + ^-^n g 

2.1 Large deviations I 

In the following Lemmas we bound Ej^, for z/ = 1, 2, 3 and j = 1, . . . ,n. 

Lemma 2.1. Under the conditions of Theorem \l.l\ we have, for any z = u + iv 
w > and any j = 1, . . . ,n, 

, , 1 
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Proof. It is straightforward to check that 

p+n 



p — n 

z 

1=1 1=1 



^ Ri+n,i+n = nmn{z) = ^ i?// - nmn{z) (2.7) 



and 



Furthermore, 



p p+n 



(=1 i=i,i¥'j 



nm„(z) = ^TrR + (n - l)m^^K{z) = ^Tr R(^) + (2.9) 



This implies 



E - E <n,/+n = ^Tr R - iTr R(^) - - (2.10) 



The conclusion of Lemma 12.11 follows immediately from the inequality | Tr R — 
TrR(J')| < ir^ and |i| < V ^ (see Lemma 4.1 in [3]). □ 

Lemma 2.2. Assuming conditions of Theorem \l.l\ for any a > there exist positive 
constants C and c, depending on a and x only such that for any f > 0, the following 
inequality holds 

p 

Pr{|e,i| > 2n-hiX^{n-'Y.\^^+nHn?y^} < Cexp{-c/„,J 

1=1 

Proof. The proof of this Lemma is similar to the proof of Lemma 2.3 in [6]. We use 
the following inequality for the sum of independent random variables. Let C,i, ■ ■ ■ ,C,p 
be independent random variables such that E^j = and < aj. Then 

P 2 

Pr{| E^^l > ^} ^ - "^(^Z^) ^ -exp{-f^}, (2.11) 



where $(x) = J^^ exp{ — ^}dy and a'^ = af -\ h CTp. We put, for /c = 1, . . . ,p, 

rjk = Xjf^ — 1, and define 



Note that E^^ = and \C,k\ < 2/n,a|-Rfc+„ For any j = 1, . . . ,n, introduce the 
(T-algebra ^^■'^ generated by the random variables Xik with 1 < I j < n,l < k < p. 
Let Ej and Pr,- denote the conditional expectation and probability given Note 
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that random variables Xjk and a- algebra are independent. Applying inequality 

with X := 2nkl,f- (n^' ELi l4+n„.+nP) we get 

P P 2 

Pr{| 5^61 >x} = EPr,{| ^^fcl > x} < Eexp{-^} < Cexp{-c/„,4. (2.12) 

k=l k=l 

Furthermore, note that, for k = 1, . . . ,p, 

|E,r/fcI{|6l < < E||r/fe|2pr|{|a-| > < Ir/^p exp{-|/„,4. (2.13) 

The last inequality implies that 

lX:E,r/a{|X,.| </|4)i?g„_,^J <Cexp{-|/„,J(i^^ (2.14) 

fc=l leTj 

The inequalities fl2.12p and fl2.14p together conclude the proof of Lemma 12.21 Thus 
the Lemma is proved. 

□ 

Corollary 2.3. Under the conditions of Theorem \l.l\ there exist constants c and C 
depending on a and k only such that for any z = u + iv with w > 0, 

Pr{|£,i| > Pl{nv)--^{\mm^^{z) + ^^"f"" )"} < Cexp{-cZ„,J. (2.15) 

Proof. Note that 

-y\B!iLk+n? < -Tr |R(^f = — ImTrR(^) < -Immi^)(^) + ^"""^^ Im-, 

k=l 

(2.16) 

where jR^-'^p = R(j)r(-')*. The resuh follows now from Lemma [2.111 □ 

Lemma 2.4. Under the conditions of Theorem \l.l\ for any j = 1, . . . ,n and for any 
V > , the following inequality holds 

Pr{|.,2|>/3>-i(^ Yl \Rkl,i^n\'y}<CeM-cln,o.}. (2.17) 

l<kj^l<p 

Proof. We shall use a bound for the large deviation probabilities of quadratic forms 
which follows from results of Ledoux (see |9]). 

Proposition 2.1. Let ^i,...,C,p be independent random variables such that < 
1. Let also aik be complex numbers such that aik = aki and akk = 0. Let Z = 
k=i iiikO-ik- Let (T^ = ^ffc^i Iflifcp- Then for every t > there exists some positive 
constant c > such that the following inequality holds 

Pr{|Z| > -E5|Z|2 + t} < exp{--} (2.18) 
2 a 
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Proof. The result 12.11 follows from Theorem 3.1 [9]. □ 
In order to bound ej2 we use Proposition 12.11 with 

Cfc = (-^jfcl{|^jfc| < ln,a} — EXjiJ[{\Xjk\ < ln,a})/'2ln,a, (2.19) 

for k = 1, . . . ,p. Note that the random variables Xjk, k = 1, . . . ,p and the matrix 
R'^-'^ are mutually independent for any fixed j = 1, . . . ,n. Moreover, < 1. Put 
^ '■= Y.i<k^i<piiikRu ■ Applying Proposition O with 

t := n5/„,„(n-i T.i<i^k<p 1^!? ^e get 

EPr,{|Z| > ^ li^Jf^l^)^} < Cexp{-c/„,J. (2.20) 

Furthermore, 

Pr{3j G [l,...,n]and3A; G [l,...,j)] : > < C exp{-c/„,c.} (2.21) 
and, for any /c = 1, . . . , p, 

\EXjkl{\Xjk\ < lia}\ < FtHBj G [l,...,n], A;G [l,...,p] : \Xjk\ > /|,«} 

< Cexp{-c/„,a}. (2.22) 

Introduce the random variables 

11 ^ 

'■= XjkI{\Xjk\ < ln,a}/'2ln,a and Z := ^ ^likR^+n,k+n- 

l,k=l 

Note that 

p 

Pr{ 5^ X,,X,,i?g„^,^„ ^ Z} < C7exp{-c/„,J. (2.23) 

Inequalities (I22DD-(I223D together imply 

Pr{|5,2|>/|;V^(i 5^ |41,^j2)^}<Cexp{-c/„,J. (2.24) 

i<fc^i<p 

Thus, Lemma 12.41 is proved. □ 

Corollary 2.5. Under the conditions of Theorem \l.l\ there exist constants c and C, 
depending on x and a such that for any z = u + iv with f > 

Pr{|e,2| > (3l{nvr'^{lmmi^\z) + ^^"tp^^"" )'} < exp{-c/„, J. (2.25) 
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Proof. Note that 

' E < n-'Tr |R(^Y < 'lmm(f)(^) + " " + (2.26) 



n 

i<fc^i<p 



The resuh follows now from Lemma [2. 4[ 



□ 



Collecting these results recall the definition 

-+- 

Pn ~ ln,a ■ (2.27) 

Then we may write, for v = 1, 2, 3 



Denote by 



(]„(.).= ^ c G . |.,| < -j= ( + + ) ^ (2.29) 



Let fo — with sufficiently small positive constant d. We introduce the region 
V := {z = u + iv E C : \u\ & Je, vq < v < 2} and a sequence Zi = Ui + Vi in V, 
defined recursively via n^+i — ui = ^ and t^^+i — = We introduce the events 

JiU.,) := e : < ^ (^^^^^ + ' + ^H^) }. (2.30) 

L y/n\ y/v y/nv \z\ J) 



Using a union bound we obtain 

Pr{n,,ei,f]'„(z/)} > 1 - Cexp{-c/„,J. (2.31) 
It is straightforward to check that 

\a(.\ ^ ^ \^-^'\ ^ (Imm„(2))i 1 , , s 

This immediately implies that, for \z — zi\< 

^n{z) C (^U^O (2.33) 

and 

Pr{n,e2,l],(^)} > 1 - Cexp{-c/„,J (2.34) 
with some constants C and c, depending on a and x only. 
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3 Large deviations II 

In this Section we obtain bounds for the large deviation probabilities of the sum of 
the Ej. We start with the quantity 



^- = ;j^EE(^^-l)<n,^+n- (3.1) 

j=l k=l 

We prove the following Lemma 

Lemma 3.1. Assume that there exists a constant Cq such that for any j = 1, . . . ,n 
and any k = 1, . . . ,p 

lmml^\z)<Co. (3.2) 

Then there exist constants c and C, depending on a, x and Cq such that 

Pr{|5„i| < n-^v-'^pl} < Cexp{-cZ„,„} (3.3) 

Proof. For any j = 1, . . . ,n and any k = 1, . . . ,p, we introduce the truncated random 
variables 

= X%- EX%, (3.4) 
where Xj^ = XjkI{\Xjk\ < ln,a}- It is straightforward to check that 

< 1 - EXff^ < Cexp{-cln,a}- (3.5) 
Introduce as well the quantities 

j=l k=l j=l k=l 

By assumption (11. 2p . 

PT{6nl 7^ Snl} < C exp{-cln,a}- (3.7) 

Let 

1 ^ 

•= E ^JkRk+n,k+n- (3-8) 
^ k=l 

Then 

1 " 

^ni = -2Y.(r (3-9) 

i=i 

Note that the sequence 5ni is a martingale with respect to the cr-algebras SHj. In fact, 

E{Cj\^j^i) = E(E{Q\^^^^)\mj^i^ = 0. (3.10) 
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In order to use large deviation bounds for 6ni we replace the differences Q by truncated 
random variables. Introduce 



6 :=oi{ioi (3.11) 

k=l 

Since (j is sum of independent bounded random variables with mean zero, we have 



k=l 



This implies that 

n n 

j=i j=i 

Furthermore, introduce the conditionally recentered random variables 

Q = Q-E{^m,^^}. (3.14) 

Using the Cauchy-Schwarz inequality and boundedness of the random variables ^jkR^i+n k+n 
it follows that 

|E{0|a«,-i}| < Cexp{-c/„,„}. (3.15) 

Using a martingale bound by Bentkus, [2], Theorem 1.1, we obtain the following 
result. Let SHq = {0, fi} C SUi C ■ ■ ■ C StJn C 5){ be a family of sub- a- algebras of the 
measurable space {^2, 5K} and let M„ = + . . . + ^„ be a martingale with bounded 
differences = Mj — Mj_i such that 

Pr{|e,| < = 1 for j = l,...,n. 

Then, for x > a/8, 

Pr||M„|>x|<c(l-$(-)) = y^ ip{t)dt, y,(t) = -^exp{ — }. (3.16) 

(7 

with some numerical constant c > and a"^ = h\ + ■ ■ ■ + h\. Note that for t > C 

We shall use now the inequality 

Pr{|M„| >x} <exp{-^} (3.17) 

to bound 5ni- By assumption (13. 2p and the definition of we may choose Pj = ln,a ^ , 
obtaining 

Pr{|5„i| > n-\-hm'^m^i\z)l^X^} < Cexp{-c/„,,}. (3.18) 
Inequalities f l3.13p -f l3.18p together conclude the proof of Lemma 13. 1[ □ 
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Let 

1 " 

'^"2-=^X] ^il^jkRl+n,k+n- (3-19) 

j=l l<l^k<p 

Lemma 3.2. Assume that there exists a constant Co such that for any j = 1, . . .n 

lmml^\z) < Co. (3.20) 
Then there exist constants c and C, depending on a, x and Co such that 

Pr{|5„2| > -^Pllm'^mli^z)} < Cexp{-c/„,J (3.21) 



n\/v 



Proof. The proof of this Lemma is similar to proof of Lemma 13.11 We introduce the 
random variables 



$Z ^jkXjlR'lin,k+n (3-22) 

and note that the sequence 



Vj 

A / 7 /, 

l<l=^k<p 



Mn = J2vj (3.23) 

i=i 

is a martingale with respect to the cr-algebras We now apply the martingale 
large bound f l3.17p twice, replacing rjj by truncated random variables. Thus Lemma 
is proved. □ 



Finally we have to bound 

n p 



^ n p 

'^"3 •= ~^ XI ^iRk+n,k+n " Rkln,k+n) ^jj ■ (3.24) 

I b 

3=1 k=l 

Lemma 3.3. There exists a positive constant C such that 

ISnsl < — Imm„(2;). (3.25) 

nv 

Proof. It is easy to check that 

X^(i?.+n,.+n - 41,.+n) = ^(T^R " Tr R^^^) + ^ (3.26) 
fc=l 

By formula (5.4) in [3], we have 

1 . rl 

(TrR - Tr R(^))i?,, = (1 + - ^ X,,X,,(i?(^))2),+„,,+„)i?2^. = -^Rjr (3.27) 

^ Lk=l ^ 
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From here it follows that 

1 " 1/7 

- V(TrR-TrR(^))i?,,. = ---m„(^). (3.28) 



Note that 



and 



mn{z)='-±^, (3.29) 



d , . mJz) 2z^ A 1 



dz'^-^'^ = — V^(?37^- ('-'O) 



This imphes that 



^2 " 



t— • (St - 2 

fe=i 



Finally, we note that 

1 1 



{s\ — z'^y 



< — lm.mn{z). (3.32) 
nv 



The last inequality concludes the proof Thus Lemma 13.31 is proved. □ 
3.1 Stieltjes transforms 

In this section we derive auxiliary bounds for the difference between the Stieltjes 
transforms m„(z) of the empirical spectral measure of the matrix V and the Stielt- 
jes transform Sy{z) of the symmetrized Marchenko-Pastur law. We introduce the 
additional notations 



^ 'i n 

= Snl + Sn2, = ^nSi = — ''^^ ^ ju^ j ^jj ■ (3.33) 

Recall that Sy{z) satisfies the equation 

^y(^) = -— hriFT- (3-34) 

z + ysy{z) + ^ 

Denote by gn{z) = mn{z) — Sy{z). The representation (12. 6p and the equality (13.340 
together imply 



ygn[Z)Sy[Z) On 

z + ymn{z) + ^ {z + yrriniz) + 



+ r + ^. (3.35) 

z + ymniz) + i^ {z + ymr,{z) + 
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Introduce the following notations 

y - 1 

hn{z,y) = z + ymn{z)^ , and an{z,y) = hn{z,y) + ySy{z). 

z 

From equaUty fl3.35p it follows that 

\9n{z)\ < |, '^"'^'y + J?^. (3.36) 

\bn[z,y)\\an[z,y)\ \an[z,y)\ 



For any z (iV introduce the events 



;^ / X r ^ , r , dnlYa2mn\z)-\ ~ , , f ^ lmm„(z) 
I n\/v ) I 



nyt; -I I. nv J 

(3.37) 

S„(^) = (c. € n : |5„| < (^>!M£) + ^ + ^ (3.38) 

Put ^*rX.z^ = Quiz) UQniz) U ^Iniz) . By Lemmas I3.m3.3t we have 

Pr{nn{z)} > 1 - Cexp{-c/„,„}. (3.39) 



By Lemma 13.31 

Pr{fi„(z)} = 1. (3.40) 

Note that 

\ej.e,s\<\i\e,.\' + \e,s\'). (3.41) 
By inequality fl2.28p . we have, for u = 1,2, 

p r > « (IEIM£) ^ ^ ^ I ^ cexp{-d„,„}. (3.42) 
Furthermore, 

Pr{k,3r < ^} = 1- (3.43) 
Similar to equality f l2.34p we may show that 

Pr I n.ei, {n:Xz) n n^iz)) } > l - C exp{-c/„, J. (3.44) 

Let 

n*^:=n,^T>{Ki^)nnn{z)). (3.45) 

In the what follows we shall assume that 

^0 = ^ (3.46) 

n 

with b > 32. We now prove the first essential bound. 
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Lemma 3.4. Let z = u + iv (zT). Assume that 



\9n{z)\ < \. (3.47) 



Then for any u E fl^, the following bound holds 



nv ra^f 2^/7 + V 
Proof. First note that the inequahty |5'„(2;)| < | imphes 

y — 1 y — 1 1 

\bn{z,y)\ = \z + ymn{z) H 1 > \z + ysy{z) H 1 - \ygn{z)\ > -. (3.48) 

Moreover, by definition of Vq f l3.46p . we have 

Imm(f)(^) < Wi\z)\ < \mn{z)\ + — < \sy{z)\ + \gn{z)\ + — < 2. (3.49) 

nv nv 

Furthermore, we have the obvious inequality 

\an{z,y)\ = \z + ymn{z) +ysy{z) + > Im(2 + ^—^ + ysy{z)) 



> ^Im U {z + ^)2 - 4|/) . (3.50) 



Moreover, 



K(.,.)|>Im(i-i)=^. (3.51) 



For 2 e P we obtain Re ((^ + - 4i/) < and f < arg((^ + - Ay) < ^. 
Therefore, 



Im (^(2; + ^)2 - 4y) > i= I (2 + ^)2 - 4y I ' > ^VlT^, (3.52) 

where 7 = min(l + y/y — \u\, 1 — y/y — \u\). Inequality (I3.36P implies that for a; G f2* 
, ,, Pn Im m„(2;) 



n^/v\bn{z,y)\\an{z,y)\ nv\an{z,y)\ 

Furthermore, equation (12. 4p . inequality fl3.48p and definition of fi* in fl2.30p together 
imply that, for a; G fi* and z eV 



The rate of convergence to the Marchenko-Pastur distribution 



15 



Inequalities f l3.49p -( l3.54p together imply 



\9n{z)\ < ^ ; ^ 1 + T= + 



n^/v\aniz,y)\\ ^/u nv 

nv\an{z,y)\ nv\an{z,y)\ 
< M ^ ^J^, (3.55, 

This inequality completes the proof of the Lemma. □ 

Put now v'q := Vq{z) := where 7 = min{|u| — 1 + ^/y, 1 + y/y — \u\ and 
z = u + iv. Denote V := {z G V : v > v'q}. 

Corollary 3.5. Assume that for w G f2„ and z = u + iv ^ V, 

\gn{z)\ < ^ holds. 



Then 



Proof. Note that for v > Vq 



i^S+_|H^<i (3,56) 

nv n^V'^^'^ + f 4 

Thus the Corollary is proved. □ 
Corollary 3.6. Let z = u + iv eT). Assume that 

\hn{z,y)>\. (3.57) 
Then for any w G fi* , the following hound holds 



nv n^v^^J^ + V 

Proof. The proof of Lemma 13.41 relies on the condition fl3.47l) of Lemma 13.41 only to 
prove inequality f l3.57p . Thus the Corollary is proved. □ 

The next lemma yields a recursion which plays a crucial role in our proof lemma 
3.4 and is similar to an approach used in [7]. 



Lemma 3.7. Assume that for some z = u + iv E V with v > Vq condition ( [J.^Tp 
holds. Then it holds for z' = u + iv' G T) such that Q < v — v' < n~^ and v' > Vq as 
well. 
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Proof. First of all we note that 

\m4z) - m„(z')| = -{v - t;')|TrR(^)R(^')l < ^ < 7^ < I (3-58) 

n vv' ¥p^n° 8 

and 

\sy{z)-Sy{z')\<^- (3.59) 

By Corollary 13. 6t we have 

\9n{z)\<\. (3.60) 

All these inequalities together imply 

l^?n(^')l<^- (3.61) 

Thus the Lemma is proved. □ 
Proposition 3.1. There exists constants C , c, such that 

Vr{\g^{z)\ > ^ + ^ < Cexp{-c/„,J. (3.62) 

Proof. Note that for f = 2 we have 

y -I 

\z + ymn{z) H 1 > Imz > 2. (3.63) 

By Lemma [3.41 we obtain the inequality fl3.62p . By Lemma [3.71 this inequality holds 
for any 2>v> Vq. Thus proposition 13.11 is proved. 

□ 



4 Proof of Theorem 1.1 



To conclude the proof of Theorem 1 1.1 1 we modify a bound for the Kolmogorov distance 
between distribution functions based on their Stieltjes transforms obtained in |3], 
Lemma 2.1. Given e > introduce the interval = [1 — ^Jy + £, 1 + ^Jy — e\ and 
= [1 — + \e,l + y/y — }^e\. For any x such that |x| G define 7 = 7(x) := 
min{|x| — 1 + y^, 1 + y/y — \x\. For a distribution function F denote by Sf{z) its 
Stieltjes transform, 

Sf{z) = / dF{x). 



Proposition 4.1. Let v > and a and e > be positive numbers such that 

If 1 3 
a = - — — -du = - (4.1) 
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and 



2va < e^. 



(4.2) 



// Gy denotes the distribution function of the symmetrized Marchenko-Pastur law, 
and F is any distribution function, there exists some absolute constants Ci, C2, C3 
depending on y only such that 

A{F,Gy) := sup \F{x) - Gy{x)\ 



< Gi sup 



Im 



V V 

{Sf{u + i—) - Scyiu + i—))du 



+ G2V + Cs^a. 

(4.3) 

Proof. Without loss of generahty we may assume that < y < 1. The case y = 1 
is considered in [6]. The proof of Proposition 14.11 is an adaption of the proof of 
Proposition 4.1 from [6]. We provide it here for completeness. Note that 

sup|F(x) -^^(x)! < snp\F{x)-Gy{x)\+Gy{-l-^ + e), (4.4) 

and 

Gy{-l-^ + e)<Gel. (4.5) 

Let X G Je. Recall that 7 = min{|a;| — 1 + y^, 1 + ^/y — \x\}. Then, according to 
condition (14. 2 p we have x + ^ ^ K- Denote hj v' = For any x G we have 



— Im 

TT 



{Sf{u + iv') - Scyiu + iv'))du 



> — Im 

TT 
_ 1 
TT 
1 



{Sf{u + iv') - Scyiu + iv'))du 



v'd{F{q) - Gy{q)) 



du 



{q — uy + v''^ 
^ 2v'{q-u){F{q)-Gy{q))dq 



((g-M)2 + t;'2)2 
(F(g) - Gy{q)) 



2v'(q — u)du , 



{F{x — v'q) — Gy{x — v'q))dq 



(4.6) 



Since F is non decreasing, we obtain 
1 {F{x - v'q) - Gy{x - v'q))dy 



TT ./_^ + 1 

> a{F{x — v'a) — Gy{x — v'a)) 

> a{F{x — v'a) — Gy{x — v'a)) 



\Gy{x — v'q) — Gy{x — v'a))\dq 



kl<a 



\Gy{x — q) — Gy{x — v'a))\dq. 

(4.7) 
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Denote by A,{F,G) 

— sup^gj^ Gy(^x)\ and let Xn G such that F{xn) 

Gy{xn) — Ae(F, G). Then x„ = x„ + v'a G J'^. We have 



sup 



Note that 



Im / {Sp{u + iv) — Scyiu + iv'))du 



> a{F{xn) - Gy{Xn)) 



— sup — / \Gy{x + q)-Gy{x)\dq-{l-a)A{F,G). (4.8) 

T^V xej^ ^/l J\q\<2v'a 



1 1 

— sup — 

xar^ yjl J\q\<2v'a 



Gy{x + q) - Gy{x)\dy 



1 1 



< — sup v/4-x2 < Gv. (4.9) 



Inequahties ( 14. 4p . (14. 8 p and (14. 9 p together imply 



sup 

xel' 



Im / {Sf{u + iv') — Scyiu + iv'))du 



> (2a - 1)A,(F, G)-Gv- Ge 



(4.10) 

Similar arguments may be used for the sequence Xn G such F(x„) — Gy{xn) 
—Ai,{F,G). This completes the proof. □ 

Corollary 4.1. Under the conditions of Proposition ^^ for any V > v, the following 
inequality holds 



sup 



(Im (S'ir(u + iv') — Scyiu + iv'))du 

oo 

< / ISpiu + tV)- SGJu + tV)\du 



sup 



V 



{Spix + m) - Scyix + m)) du 



(4.11) 



Proof. Set z = n + zf'. f' < 2. Since the functions of Sf{z) and Scyiz) are analytic 
in the upper half-plane, it is enough to use Cauchy's theorem. We can write 

/oo px 
ImiSpiz) - SGyiz))du = \im {Sf{u + tv') - Scyiu + tv'))du. (4.12) 

Since v' = ^ < without loss of generality we may assume that v' < 2. By 
Cauchy's integral formula, we have 

(Spiz) - SGy{z))du = r {Sf{u + iV) - SGy{u + iV))du 



I {Sf{-L + iu) - Scyi-L + iu))du 

Jv' 



V 



{Sf{x + iu) — Scyix + iu))du. (4-13) 
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Denote by .^(?7) a random variable with distribution function F{x) {Gy{x)). Then we 
have 



\Sf(—L + iv' 



E 



1 



^ + L-iv' 



<v' 'Pr{|e| >L/2} + 



Similarly, 



Sg,{-L + zv')\<v'~'PtM > L/2} + y. 



These inequalities imply that 



{Sf{—L + iu) — Scyi—L + iu))du 



L 



— 7- as L — !■ cxD, 



(4.14) 
(4.15) 

(4.16) 
□ 



which completes the proof 

Combining the results of Proposition 14.11 and Corollary 14.11 we get 
Corollary 4.2. Under the conditions of Proposition |^ . i] the following inequality holds 

/oo 
\Sf{u + iV) - Scyiu + iV)\du + C2V + C3£5 
-00 

nV 

+ Ci sup / \Sf{x + iu) — Scyix + iu)\du, (4-17) 
xeJ^ Jv' 

where v' = ^ with 7 = min{|x| — 1 + ^Jy, 1 + \fy — \'xW- 

We apply now the result of Corollary 14.21 to the empirical spectral distribution 
function J-^„(x) of the random matrix X. First we bound the integral over the line 
with V = 1. Note that in this case we have \z + ymn{z) + > 1. Moreover, 

lmmii\z) < ^ < |. We may now apply results of the previous Lemmas on large 
deviations. This ensures the following bound for gn{z) for all z = u + iV with u G M. 



\9n{z)\ < 



/3n 



n^/v\an{z,y)br^{z,y)\ 
Note that for V = 2, 



^ ^ I3nlmmn{z) (1 - y)v \ 



({\z\' for |m| ^ [1 -yy, l + yy]. 
We may rewrite the bound ( I4.18p as follows 

Clmniniz) 



Clmmn{z) 
nv\an{z,y)\' 
(4.18) 



(4.19) 



\9n{z)\ < 



n{\z\ 



nv 



(4.20) 
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Note that for any distribution function F[x) we have 

/oo 
lmsF{u + iv)du = 71 (4-21) 
-oo 

From here it follows that, for V = 2 



|n|>n 

Denote Vn := {z = u + 2i : \u\ < n} and 



C 

niniz) - Sy{z)\du < — (4.22) 



Using a union bound, we may show that 

Pr{n„} > 1 - Cexp{-c/„,^}. (4.23) 
It is straightforward to check that for G f2„ 

/oo ^ 
\mn{z) - Sy{z)\du < —. (4.24) 

We choose e = n~3 and vq = To conclude the proof we need to consider the 
"vertical" path integrals in z = x + iv' with x & J'^, v' = ^ and 7 = 2 — It will 
be enough to consider one of these integrals only, the others being similar, namely 

2 1 1 1 /32 

dv < -—- <-—<'^. (4.25) 



Finally, we obtain for any a; G 



A{F^,G) = snp\Fr,{x)-Gy{x)\ < (4.26) 



Thus Theorem 11.11 is proved. 



5 Proof of Theorem 11.2 



Consider the singular value decomposition of the matrix X. Let U and H be unitary 
matrices of dimension nx n and p x p respectively. Let S be a n x n diagonal matrix 
whose entries are the singular value of the matrix X. Let Opxq denote the p x g-matrix 
with zero entries. Introduce the matrix S = [S 0„x(p-n)]- We have the following 
representation 

X = USH*. (5.1) 
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We may represent the matrix H in the form 



H 



Hii Hi2 
H21 H22 



(5.2) 



where Hn is n x n matrix , H22 is p x p matrix. We introduce matrix 



73^ 71 "^11 72 



O 



H 
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It is straightforward to check that 
ZVZ* = 



S 

Onxn 



o 



nxn 

s 



7I""i2 
— H12 
H22 



Onx (p— n) 

(p— n) 



(5.3) 



0(p~n)xn C)(p— n)xn C)(p— n)x (p— n) 



(5.4) 



where S denotes diagonal matrix with entries Sj. The equality (15.41) implies that 
the rows zj of the matrix Z, for j = 1, . . . ,n, are the eigenvectors of the matrix V 
corresponding to the eigenvalues sj. Similarly, the rows Zj_|_„ of the matrix Z, for j = 
1, . . . ,n, are the eigenvectors of the matrix V corresponding to the eigenvalues —sj 
and the rows Z2n+i, for / = 1, . . . , p—n, are the eigenvectors of matrix V corresponding 
to the eigenvalues 0. 

We note the following representation for the diagonal entries of the resolvent 
matrix R: 

n+p ^ 



k=l 



\Hkj\ 



(5.5) 



Denote by Ai, . . . , \n+p the eigenvalues of the matrix V ordered in such way that 



—s 



if 1 < j < n 
if n + 1 < J <2n 
0, if 2n < j < n + p. 



^31 



(5.6) 



Consider the distribution function Fnj{x) of the following weighted empirical prob- 
ability distribution on the eigenvalues Ai, . . . , \n+p 



n+p 



Y,\Hu,\^l{\u<x]. 



k=l 



Then we have 



R 



'33 



X — z 



dFjij (x) . 



(5.7) 



(5.^ 



which means that Rjj is the Stieltjes transform of the distribution Fnj{x). Note that, 
for any A > 



max \Hkj\'^ < sup{Fnj{x + A) - Fnj{x)) =: Qnj{>)- 

l<k<n+p J. 



(5.9) 
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On the other hand, it is easy to check that 

Qni(A) < 2 sup Mm Rjj{u + iX). (5.10) 

u 

By relations ( I2.30p and ( ]2.34p . for any v > vq with t>o = ^ with a sufficiently large 
constant d, we have 

£j\ 1 



Furthermore, the representation ( 12. 4p and inequality ( 15. lip together imply, for v > vq 

Im%<|%|<a (5.12) 



This implies that 



Pr{ max liJ^^f > — ^} < C exp{-c/„ „}. (5.13) 



By a union bound the inequality (11.41) follows. To prove inequality (11. 5p . we consider 
the quantity 

'fj ■= Rjj - -hi^)^ J = 1, • • • , (5.14) 
Using equalities (12. 4p and ( 13.34p . we get 

_ Syiz)g4z) sj 
By definition of H we obtain 



Pr{ max > ^-^} < Cexp{— c/„ „}. (5.16) 

l<j,k<n n 



and 



Pr{ max {vkjl"^ > < Cexp{-c/„,Q}. (5.17) 



By inequalities (13.62p and ( 12.34p . we have 



|r,| < (5.18) 
\/nv 



This implies that 



f c 

sup / \rj{x + iv)\dv<—=. (5.19) 

xeJe Jv' V IT' 

Similar to ( 14.24p we get 

\rj{x + iV)\dx < (5.20) 
00 v'^ 
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Applying Corollary 14.21 we finally obtain 



Pr{sup \Fnj{x) - Gy{x)\ < > 1 - Cexp{-c/„,4. (5.21) 



In view of ^ 

Pr{sup |F„(x) - Gy{x)\ < ^} > 1 - Cexp{-c/„,4, (5.22) 

X ^ 

we get 

Pr{sup |F„,(x) - Gy{x)\ < ^} > 1 - Cexp{-c/„,„}. (5.23) 



The last two inequalities together imply that 

Pr{sup |F„,(x) - Fn{x){x)\ < ^} > 1 - Cexp{-c/„,J. (5.24) 
X \rn, 



Note that Fn[x) is the distribution function of a random variable which is uniformly 
distributed on the set {±si, . . . , ±s„} and 



sup \Fn^{x) - Fn{x)\ = max | ^ {ukjl"^ . (5.25) 

^ 1=1 ^ 

Thus Theorem 11.21 is proved. 
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